Investigation of methods for the optimal selection of students into specialist science secondary schools

The mission of specialist science secondary schools (SSSS) is to increase the number of students with aspirations to participate in higher education courses and careers in Science, Technology, Engineering, and Mathematics (STEM; Means et al., 2021). Although many SSSS use selective admission criteria (Erdogan & Stuessy, 2015), the efficacy of such entry assessment still requires exploration. This research is investigating the characteristics of students that may best benefit from the education provided through SSSS to in-turn inform the best design for the entry assessment. Our conceptual framework (shown below) was developed from social cognitive career theory, explaining factors influencing individuals' academic or career choices (Lent et al., 1994). Based on this theory, there are four student characteristics our work has chosen to focus on: critical thinking, creativity, science self-efficacy, and science identity. These four characteristics are being explored through a validated questionnaire constructed from four pre-existing and validated instruments (Lin & Tsai, 2013; Lockhart et al., 2022; Runco et al., 2001; Sosu, 2013). The questionnaire was distributed and data were collected from students (n = 87), alumni (n = 193), teachers (n = 23), and school administrators (n = 3) from one SSSS in Thailand from April to June 2023. Participants’ demographic information was collected to investigate any correlations between cohorts and their questionnaire responses. Additional qualitative data were obtained through both open-ended responses and interviews (teacher n = 14 and school administrator n = 1) to reveal further depth and help understand participants’ perspectives. This presentation will share the questionnaire findings from one school involving this study. These preliminary results provide practitioners and decision-makers with some initial insights about the student characteristics they should focus on for the selection of suitable students for SSSS. REFERENCESErdogan, N., & Stuessy, C. L. (2015). Modeling Successful STEM High Schools in the United States: An Ecology Framework. Online Submission, 3(1), 77-92.Lent, R. W., Brown, S. D., & Hackett, G. (1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of vocational behavior, 45(1), 79-122.Lin, T. J., & Tsai, C. C. (2013). A Multi-dimensional instrument for evaluating Taiwanese high school students’ science learning self-efficacy in relation to their approaches to learning science. International Journal of Science and Mathematics Education, 11, 1275-1301.Lockhart, M. E., Kwok, O. M., Yoon, M., & Wong, R. (2022). An important component to investigating STEM persistence: the development and validation of the science identity (SciID) scale. International Journal of STEM Education, 9(1), 1-17.Means, B., Wang, H., Wei, X., Young, V., & Iwatani, E. (2021). Impacts of attending an inclusive STEM high school: meta-analytic estimates from five studies. International Journal of STEM Education, 8(1), 1-19.Runco, M. A., Plucker, J. A., & Lim, W. (2001). Development and psychometric integrity of a measure of ideational behavior. Creativity Research Journal, 13(3-4), 393-400.Sosu, E. M. (2013). The development and psychometric validation of a Critical Thinking Disposition Scale. Thinking skills and creativity, 9, 107-119

Establishing online examination guidelines: Preliminary results from an ACDS Teaching and Learning Project 2022

In recent years, many universities in Australia and worldwide have seen major changes to their teaching and learning delivery. This includes assessment strategies, an example of which are summative final examinations. Historically, closed-book, in-person, paper-based final examinations were commonly used across the sector (Williams & Wong, 2009). However, during the COVID-19 pandemic many universities moved from traditional paper-based examinations to online delivery (Dicks et al., 2020). Online examinations have been delivered in a variety of formats, and with different implementations. Thus, we are at an opportune time to re-evaluate assessment for and of learning to ensure that we make pedagogically informed changes and establish robust procedures moving forward. In this research, funded by an Australian Council of Deans of Science (ACDS) Teaching and Learning Project grant 2022, we present the preliminary results of a multi-institution exploration of first-year undergraduate examinations in STEM subjects comparing end-of-semester examinations from 2019–2021. To determine the pedagogical changes that occurred, we undertook a multi-step analysis of: i) Question type and format; ii) Order of thinking pattern required to respond to questions (Agarwal, 2019); iii) Classification of question according to Bloom’s Taxonomy (Bloom et al., 2001); iv) Level of abstraction. Outcomes from our data analysis will inform practitioners and decision-makers on best practices whilst balancing university and student expectations, with delivering authentic assessment experiences. Our research is enabling us to make meaningful recommendations for best practice in Australian STEM subjects for summative examinations, including design that considers both technological as well as pedagogical aspects required to deliver effective assessments. REFERENCES Agarwal, P. (2019). Retrieval Practice & Bloom’s Taxonomy: Do Students Need Fact Knowledge Before Higher Order Learning? Journal of Educational Psychology, 111(2), 189–209. https://doi.org/10.1037/edu0000282. Bloom, B. S., Airasian, P., Krathwohl, D. R., Cruikshank, K., Mayer, R., Pintrich, P., Raths, J., & Wittrock, M. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Anderson, L. W., Bloom, B. S., Krathwohl, D. R., (Eds.), Longman: New York. Dicks, A. P., Morra, B., & Quinlan, K. B. (2020). Lessons learned from the COVID-19 crisis: Adjusting assessment approaches within introductory organic courses. Journal of Chemical Education, 97(9) 3406–3412. https://doi.org/10.1021/acs.jchemed.0c00529. Williams, J. B., & Wong, A. (2009). The efficacy of final examinations: A comparative study of closed-book, invigilated exams and open-book, open-web exams. British Journal of Educational Technology, 40(2) 227–236. https://doi.org/10.1111/j.1467-8535.2008.00929.x

STUDENT PERCEPTION OF PREPAREDNESS IN THE MIDST OF COVID-19: A SNAPSHOT FROM FIRST YEAR CHEMISTRY STUDENTS

The transition from high school to tertiary education can present many challenges for students. First year students must navigate new formal curricular, societal norms, physical environments, and support networks. An important factor for a successful transition from secondary to tertiary education is student preparedness. This transition period was thrown on its head due to the global challenges that the COVID-19 pandemic presented in 2020. The aim of this research was to identify and examine the perceptions of preparedness of first-year chemistry students, and if these perceptions were significantly impacted by COVID-19. Surveys were deployed to a first-year chemistry cohort at both the start and the end of semester, and follow up focus groups conducted after the conclusion of the semester. Results indicate that students’ perceptions of preparedness for studying chemistry increased over the course of the semester, however for studying at university in general the perception of preparedness decreased. The absence of in-person laboratory practicals was found to be a great concern for students, along with factors previously found to impact students’ perceptions. Reflections and findings of the students’ experiences will be presented from the first semester of 2020 delivered online through emergency remote teaching and learning

Real-world connections to sustainability: Using authentic learning activities to introduce students to systems thinking through green chemistry

Systems thinking refers to approaches to learning that emphasise the interdependence of components in dynamic systems and how they interact and influence one another (Mahaffy et al., 2019). Applying systems thinking to green chemistry teaching and learning can create a molecular basis for sustainability (Mahaffy et al., 2019) that is able to enhance undergraduate chemistry students’ multidimensional understanding of complex sustainability challenges (Smith, 2011). However, efforts to introduce sustainable systems thinking – specifically within first-year introductory chemistry courses – are particularly challenging, and past approaches have produced mixed success (Mahaffy et al., 2019; An et al., 2021). Consequently, this indicates an opportune space within undergraduate chemistry education research to explore alternative and multidisciplinary approaches towards teaching green chemistry and sustainability (Wissinger et al., 2021). In this research, we present the preliminary results of a trimester-long intervention using authentic learning activities to introduce first-year chemistry students to systems thinking, through the application of green chemistry concepts. To determine the effectiveness of the intervention, we are using a mixed-methods research design to assess the impact of the learning activities on students’ development of systems thinking skills. Student motivations and attitudes towards the subject of chemistry will also be evaluated via validated survey instruments (Guay et al., 2000; Liu et al., 2017). The learning activities have been designed and developed successfully, though the delivery of the intervention is currently ongoing. Preliminary results indicate that students are excited to learn about how chemistry can be more sustainable, and that they are engaging with the learning activities. The aim of this research is to provide rigorous evidence for using systems thinking as a tool to teach students about green chemistry, ‘future-proofing’ chemistry in a way that is relevant, meaningful, and authentic for today’s chemistry students. Outcomes from our data analysis will help inform the development of new undergraduate chemistry education curricula that align with contemporary sustainable challenges. REFERENCES An, J., Loppnow, G.R., & Holme, T. A. (2021). Measuring the impact of incorporating systems thinking into general chemistry on affective components of student learning. Canadian Journal of Chemistry, 99(8), 698–705. Fisher, M.A. (2019). Systems thinking and educating the heads, hands, and hearts of chemistry majors. Journal of Chemical Education, 96(12), 2715–2719. Guay, F., Vallerand, R. J., & Blanchard, C. (2000). On the assessment of situational intrinsic and extrinsic motivation: The Situational Motivation Scale (SIMS). Motivation and emotion, 24(3), 175–213. Liu, Y., Ferrell, B., Barbera, J., & Lewis, J. E. (2017). Development and evaluation of a chemistry-specific version of the academic motivation scale (AMS-Chemistry). Chemistry Education Research and Practice, 18(1), 191–213. Mahaffy, P. G., Matlin, S. A., Holme, T. A., & MacKellar, J. (2019). Systems thinking for education about the molecular basis of sustainability. Nature Sustainability, 2(5), 362–370. Smith, T. (2011). Using critical systems thinking to foster an integrated approach to sustainability: A proposal for development practitioners. Environment, development and sustainability, 13, 1–17. Wissinger, J. E., Visa, A., Saha, B. B., Matlin, S. A., Mahaffy, P. G., Kümmerer, K., & Cornell, S. (2021). Integrating sustainability into learning in chemistry. Journal of Chemical Education, 98(4), 1061–1063

Leveraging natural language processing for comprehensive studies of science student projects

Student research projects are a crucial part of the Australian and New South Wales (NSW) High School Curriculum. In NSW, the extension science course offered for the Higher School Certificate is an example of an extensive project performed by students. The objective of the course is to provide students the opportunity to authentically apply scientific research skills. Extension science and related courses for high school students are commonly assessed through scientific reports submitted as a final summative assessment (Science Extension | NSW Education Standards, n.d.). This gives rise to large volumes of disparate data which can potentially be analysed for insights to improve science teaching and learning. Understanding these insights are especially important for priority groups to increase accessibility and equity and reduce academic attainment gaps in science. Previous research analysing student projects has been limited to studying small numbers of projects, due to the availability of data and the time taken for manual data analysis. This also limits analyses to single diversity variables, such as ethnicity (Carlone & Johnson, 2007). There is an opportunity to be realised in the data from student projects that may inform how teachers can better cater for the needs of students in various priority groups moving forward. This study outlines a method to address this research gap, by employing artificial intelligence (AI) capabilities, particularly natural language processing (NLP) techniques, to examine large sets of science high school students' final project reports such as those retained by student science fairs. A range of AI techniques have been evaluated to enable us to process and analyse sizable datasets to explore the rich information they contain. NLP techniques have been developed to classify and analyse projects along various dimensions, such as the alignment with the Field of Research (FoR) codes, the research themes. The dimensions identified will then be analysed and correlated with demographics relating to priority groups. These methods are informing the development of a reliable and repeatable AI-powered framework to analyse research themes, amongst other variables contained within science students’ final project reports. The goal of this framework is to inform the learning design of science projects to increase accessibility, student engagement and inclusion. REFERENCES Carlone, H. B., & Johnson, A. (2007). Understanding the science experiences of successful women of color: Science identity as an analytic lens. Journal of Research in Science Teaching, 44(8), 1187–1218. https://doi.org/10.1002/tea.20237  Science Extension | NSW Education Standards. (n.d.). Retrieved 22 May 2023, from https://educationstandards.nsw.edu.au/wps/portal/nesa/11-12/stage-6-learning-areas/stage-6-science/science-extension-syllabus

One-pot near-ambient temperature syntheses of aryl(difluoroenol) derivatives from trifluoroethanol

Difluoroalkenylzinc reagents prepared from 1-(2’-methoxy-ethoxymethoxy)-2,2,2-trifluoroethane and 1-(N,N-diethylcarbamoyloxy)-2,2,2-trifluoroethane at ice bath temperatures, underwent Negishi coupling with a range of aryl halides in a convenient one pot procedure. While significant differences between the enol acetal and carbamate reagents were revealed, the Negishi protocol compared very favourably with alternative coupling procedures in terms of overall yields from trifluoroethanol

PROMOTING INCLUSION IN ONLINE FIRST-YEAR CHEMISTRY THROUGH THE IMPLEMENTATION OF THE UNIVERSAL DESIGN FOR LEARNING FRAMEWORK

BACKGROUND The Universal Design for Learning (UDL) framework promotes inclusion by minimising barriers against, and maximising opportunities for learning. Implementing the three principles of the UDL framework (providing multiple means of representation, action and expression, and engagement) through its 31 checkpoints, provides strategies that allow diverse learners optimal participation in a meaningful and challenging learning environment. AIMS This paper will present an exploratory multiple-case design implementing UDL in first-year chemistry courses at two universities in Australia and one in the Philippines. DESIGN AND METHODS The UDL framework was integrated in the design and delivery of five chemistry topics, namely, periodic table and trends, chemical bonding, Lewis structures, molecular shapes, and polarity. Survey, focus groups, and interviews were conducted to gather students’ perceptions on the impact of UDL-based features in their learning. RESULTS Results from surveys, focus groups, and interviews reveal that, irrespective of their individual contexts, students from these three universities perceived positive impacts from the UDL-based features of their online chemistry learning environment. Students reported that their learning benefitted from provisions for enhanced visualisation of chemistry concepts, especially those that require chemical representations (i.e. bond formation, chemical structures, molecular geometry), improved accuracy, flexibility, self-evaluation of progress, and increased motivation. CONCLUSIONS These results suggest that applying the UDL framework in a first-year chemistry online environment can support and further enhance students’ learning irrespective of their individual contexts

EVALUATING LEARNING DESIGN OF FIRST-YEAR CHEMISTRY THROUGH LEARNING ANALYTICS

BACKGROUND Learning analytics, which involves the measurement, collection analysis and reporting of data about learners and their contexts may provide understanding and optimisation of learning environments. Recently, there has been growing interest among various education sectors in utilising learners’ data from different sources to provide support for the achievement of their specific learning goals. The expansion of online learning has yielded a rise of big data which may be employed to guide educators in designing learning environments, that together with appropriate instructional materials and methods, are able to address challenges in bridging discipline content and pedagogy. AIMS This study explored the use of learning analytics to evaluate the learning design developed for selected topics in first-year chemistry: namely periodic table, Lewis structures, types of chemical bonds, molecular shape and polarity. DESIGN AND METHODS After two weeks of online delivery of these topics to 985 learners, the log data from Moodle were collected, de-identified, processed and analysed. The aim of the analysis was to gain an understanding of learners’ interaction with the resources and activities posted on the LMS, and their online engagement with their peers and teachers. RESULTS Results from learning analytics measurements suggest that the prepared learning design afforded students not only flexible, but also independent learning, as evidenced by the usage pattern of Moodle activities over a 24-hour time frame. The log data recorded a greater frequency of access to interactive resources i.e. simulation (1721 times) and hypertext (1903 times) than the narrative resources i.e. videos (1526 times), web-based book (1561 times). This result suggests that learners choose the type of resources they perceived were most beneficial for their learning. In addition to learning resources, learners were likewise given the opportunity to select their preferred formative self-assessment activities. Results showed that more students accessed the worksheets rather than the timed quizzes. CONCLUSIONS Based on the analysis of learners’ data on their interaction with the learning resources and engagement in learning activities in the LMS, various information may be obtained to evaluate the learning design of an online first-year chemistry program

Designing evaluation for a high school research integrated learning outreach program

BACKGROUND Educational outreach programs that bridge university and high school contexts aim to increase students’ engagement and commitment to STEM career pathways (Tytler et al., 2017). Likewise, opportunities for students to engage in authentic scientific research (research integrated learning, RIL) have also been demonstrated to develop STEM skills and lead to increased STEM identity and aspiration (Beier et al., 2018; Stets et al., 2017). Outreach programs may also support access to STEM pathways for students from diverse backgrounds (Scull & Cuthill, 2010). However, these positive outcomes cannot be assumed from program design alone. Rigorous evaluation is required to ensure that goals are being reached and to support iterative improvements in program design (Australian Academy of Science, 2019). AIMS This study presents the design and preliminary results of an evaluation of a high school outreach program, SciX (this intervention is described in another ACSME 2023 abstract by Laura McKemmish). The evaluation approach focuses on the program’s impact on students’ perceived scientific skills, science identity, and commitment to pursue a career in STEM. It also explores differences in these effects between demographic groups (girls, students from rural locations, students from low SES schools). Other program goals including sustainability, scalability, and a positive impact on teachers and mentors are also incorporated. The goal of this research is to support iterative improvement of the intervention and identify transferrable principles to enhance the effectiveness and equity of similar student research programs. DESIGN AND METHODS An evaluation framework was designed around a program logic model that also incorporated elements of motivation theory and identity theory. Student surveys (University of New South Wales Human Research Ethics approved) were administered immediately before and immediately following the one-week RIL summer school in 2021-2023. A total of 238 students completed a survey following the intervention. RESULTS AND CONCLUSIONS The full evaluation framework will involve mixed methods studies that incorporate survey and interview data from students, teachers, and mentors, as well as key informant interviews and program administration data. Preliminary results will be presented from student surveys. These results show positive experiences (73% of respondents were extremely happy they attended) and significant increases in science identity, which were greater for girls. Survey results also reveal the key importance of the student-mentor relationship to the experience, which will be further explored in later studies. Initial results have informed program design and future iterations of the evaluation research. REFERENCES Australian Academy of Science. (2019). Women in STEM Decadal Plan. www.science.org.au/womeninSTEMplan Beier, M. E., Kim, M. H., Saterbak, A., Leautaud, V., Bishnoi, S., & Gilberto, J. M. (2018). The effect of authentic project‐based learning on attitudes and career aspirations in STEM. Journal of Research in Science Teaching, 56(1), 3-23. Scull, S., & Cuthill, M. (2010). Engaged outreach: using community engagement to facilitate access to higher education for people from low socio‐economic backgrounds. Higher Education Research & Development, 29(1), 59-74. Stets, J. E., Brenner, P. S., Burke, P. J., & Serpe, R. T. (2017). The science identity and entering a science occupation. Social science research, 64, 1-14. Tytler, R., Symington, D., & Cripps Clark, J. (2017). Community-School Collaborations in Science: Towards Improved Outcomes Through Better Understanding of Boundary Issues. International Journal of Science and Mathematics Education, 15(4), 643-661

Online learning in chemistry: Design, development, accessibility, and evaluation

Online learning has played an integral role in delivering large-cohort chemistry courses in undergraduate degree programs. This study includes describing how a first-year chemistry course transitioned from traditional face-to-face teaching to blended learning using the Resource-Based Learning framework (Hannafin & Hill, 2007; Reyes et al., 2022a). Using this framework, different types of online learning resources were curated to deliver chemistry content. A variety of learning activities were also developed to enhance these resources guided by Laurillard’s Conversational Framework (Laurillard, 2002). Considering that accessibility is a critical aspect to improve students’ learning experience, the Universal Design for Learning (UDL) framework was integrated into the learning design of first-year chemistry (Rose & Meyer, 2002; Reyes et al., 2022b). The perceived utility of online learning resources enhanced with UDL-based features was evaluated through students’ responses to surveys, interviews, and focus groups. Furthermore, learning analytics using temporal, sequence, and process mining analytical techniques were employed on students’ trace data to evaluate course learning design and to understand students’ engagement with learning resources and activities included in the course. Results of this study show the importance of careful development and implementation of learning design of the online learning component of chemistry courses, to enhance the students’ learning experiences. REFERENCES Hannafin, M. J., & Hill, J. (2007). Resource-based learning. In M. Spector, M. D. Merrill, J. van Merrienboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology. Erlbaum. Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technologies (2nd ed.). RoutledgeFalmer. Reyes, C.T., Kyne, S. H., Lawrie, G. A., & Thompson, C. D. (2022a). Implementing blended first-year chemistry in a developing country using online resources. Online Learning, 26(1), 174–202. https://doi.org/10.24059/olj.v26i1.2508 Reyes, C.T., Lawrie, G. A., Thompson, C. D., & Kyne, S. H. (2022b). “Every little thing that could possibly be provided helps”: analysis of online first-year chemistry resources using the universal design for learning framework. Chemistry Education Research and Practice. https://doi.org/10.1039/d1rp00171j Rose D. H. & Meyer A. (2002). Teaching every student in the digital age: Universal Design for Learning. Alexandria, VA: ASCD